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I. INTRODUCTION 



Over the past several years, the computational power of mainframes and personal 
computers has increased manyfold. Much larger and more complex optimization 
models (involving multiple scenarios) can be solved in less time. This implies a 
commensurate growth in the size of output. The sheer volume of output from these 
models is overwhelming; multiple-run comparisons involve even more output. Current 
analysis techniques have become awkward and time consuming. 

Given sufficient time, detailed output analysis for almost any model can be 
accomplished. However, decision makers usually place time restrictions upon the 
analyst. The scope of analysis usually must be narrowed down to some manageable 
size. The application of automated analysis tools offers the potential to do more 
thorough analysis in less time. 

Computer tools are necessary to automate the analysis process and manage the 
vast amount of data and output. The aim of these tools is to enhance the analysis 
process by improving the quality, speed and scope of output analysis, and reduce data 
input errors. The design of visualization tools to aid operations research analysis must 
be preceded by a careful evaluation of the decision-making process. Applicable 
visualization principles [Ref. l:p. 11-12] must be incorporated into the design. There 
are many different ways to present information, therefore, design choices must be 
made. 
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The focus of this thesis is to (1) analyze a decision-making process and identify 
analysis choke points, (2) design analysis tools, and (3) demonstrate them. 

An important operations research model, HEAVY ATTACK, has been selected 
to demonstrate these tools. This large-scale optimization model has been used by the 
Combat Forces Division at Headquarters, United States Air Force, for over two decades 
to generate air-to-ground munitions requirements for the Air Force Statement of 
Requirement in order to justify procurement objectives. HEAVY ATTACK is widely 
accepted by the Department of Defense and the Congress, and is used to help plan the 
expenditure of over $2 billion each year. Recently, HEAVY ATTACK was used by the 
Combat Forces Division for analysis of munitions requirements for Operation Desert 
Storm. 

Chapter II provides a general view of the HEAVY ATTACK model. Chapter III 
describes the current difficulties with using HEAVY ATTACK and with analyzing its 
output. Steps to improve the analysis are described. Chapter IV discusses the 
implementation of the analysis tools. These tools are based on spreadsheets. Chapter 
V states the conclusions. 
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H. HEAVY ATTACK MODEL 



For over 25 years mathematical progr ammin g has been the foundation for the 
budget planning of air-to-ground conventional ordnance by the Combat Forces Division 
within the Directorate of Operations of the United States Air Force. The pl annin g 
goal is to ensure sufficient quantities of war reserve munitions are purchased ann ually. 
The Division’s Air-to-G round Nonnuclear Consumables Annual Analysis (A/G NCAA) 
(Figure 2-1) currently uses three programs — SELECTOR, HEAVY ATTACK, and 
HEAVY GOAL — to accomplish this goal. These models reside on a mainframe 
computer in the Pentagon. 

A. A/G NCAA METHODOLOGY 

In the annual analysis process the first model, SELECTOR, determines weapons 
effectiveness based on the Joint Munitions Effectiveness Manual (JMEM) [e.g. Ref. 2], 
weather, aircraft attrition and cost. The JMEM lists the average number of targets 
of type/ killed by sortie type i using a specified tactic t in a given weather of type w. 
SELECTOR takes the JMEM data and produces, as an input to HEAVY ATTACK, the 
average number of targets of type j killed by sortie type i [Ref. 3:p. 4]. The data is 
then sorted from the most cost-effective to the least. Because the most effective 
weapon is usually the most expensive or because there may be a high attrition rate to 
the aircraft when employing a specific tactic, SELECTOR chooses the most cost- 
effective tactic. 
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The second program, HEAVY ATTACK, is the cornerstone of the planning effort. 
It is a FORTRAN-based, large-scale optimization program with a nonlinear objective 
function and linear constraints. For each of seven time periods, varying in duration 
from hours to months, HEAVY ATTACK uses the SELECTOR-nominated opt imum 
weapon for each aircraft/target combination, loads that weapon onto the aircraft, and 
allocates aircraft sorties to each target. The model’s objective is to maximize the 
expected value of targets destroyed taking into account, among other things, aircraft 
attrition, the ability of targets to regenerate themselves, weather, and Von 
Clausewitz’s "fog of war" [Ref. 4], In essence, HEAVY ATTACK fights a multi-period, 
theater-level war to determine the requirements for ordnance stock. 
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The Combat Forces Division operates these models and, for each respective 
major Air Force theater, hosts an individual annual week-long air-to-ground NCAA 
planning conference at the Pentagon. Senior staff officers bring their commander’s 
theater plan for their conference. This HEAVY ATTACK input is what the 
co mman der assesses is his mission, the threat he is facing, the enemy’s order of battle, 
and his war fighting strategy. Other data input requirements for HEAVY ATTACK 
include the sortie effectiveness data, weather, aircraft availability and weapon 
inventory. Aircraft availability, provided by the Combat Forces Division, is obtained 
from the Air Force War and Mobilization Plan, which allocates tactical air forces 
between the theaters. Weapon inventories are apportioned among the theaters by the 
Combat Forces Division. [Ref. 5] HEAVY ATTACK takes these inputs and produces 
the weapon requirements for each theater. 

When each theater has completed its annual visit, their requirements are 
combined to form an input to the last model — HEAVY GOAL. HEAVY GOAL takes 
the combined requirement, plus attrition rates for sea lift and in-theater ground 
transportation, and furnishes the aggregate weapon requirements for the Air Force 
Program Objective Memorandum (POM). 

B. HEAVY ATTACK OPTIMIZER 

In 1974 the Air Force adopted a nonlinear progr ammin g model written by Clasen, 
Graves, and Lu from RAND Corporation to "select an air-to -surface munitions mix to 
be stockpiled as war reserve inventory." [Ref. 6:p. 1] The objective was to maximize 
the "military worth" of targets destroyed by allocating sorties to targets. Their single 
period munitions mix methodology consisted of three steps: (1) selecting a munition 
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for each aircraft-target combination based on a criterion of least-cost-per-expected- 
target-killed; (2) allocating aircraft sorties to targets, with each sortie carrying the 
selected "optimum" munition; and (3) computing the required munitions mix. Figure 
2-2 shows the relationship between these steps and various inputs. 




Figure 2-2. HEAVY ATTACK Optimization Process. [Ref. 5:p. 2] 



C. INPUTS TO HEAVY ATTACK 

There are many sources of data for HEAVY ATTACK for each theater. The 
commander’s theater plan provided to the Combat Forces Division is one m^jor input; 
it is in the form of the Target Data File described below. 

Another input, the weather distribution, plays an important role in how HEAVY 
ATTACK allocates aircraft to targets. The distribution forecasts meteorological 
conditions for the theater in the form of prior probabilities for six discrete weather 
bands ranging from foul to clear weather. 
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Aircraft availability and weapons inventory constitute the remaining major inputs 
and are provided to the theater teams by the Combat Forces Division. 

Nine input Files are required for HEAVY ATTACK These files are classified as 
data, exclusion, or other. Currently, all files reside on a Pentagon mainframe 
computer. They are modified using the mainframe’s text editor. The data and 
exclusion files, which represent the majority of the data, are described in detail below. 
Since the "other" files are easily updated with the mainframe editor, they are neither 
included in the analysis, nor described. 

1. Data Files 



Targets, aircraft and weapons are defined in base data files. 
a Target Data File 



The target file (Figure 2-3) is the commander’s theater plan. There 



have been as many as 85 different target types. 
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Figure 2-3. Example of an unclassified HEAVY ATTACK Target Data File. 
The United States Air Force provided these numbers from an unclassified data 
set. The heavy dots indicate a continuation of data. 



(1) Target Name and Description. The name may be up to four 
characters and the description up to 15. Two examples are ’ABHU — AC Bkr, 
Hrd,Ugnd’ (Hardened, Underground Aircraft Bunker) and ’SA9 — SA-9 SAM battery’ 
(a Soviet-made surface-to-air missile (SAM) battery). 
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(2) Quantity. The number of targets within each target type. 

(3) Target Element. The number of elements of a target type. For 
example, each of the 40 tank parks (quantity of targets) may contain five tanks (target 
elements). 

(4) Time Weighted Target Value. A value given to each target type 
for each of the seven time periods. This value indicates how important it is to the 
commander to kill a target type relative to all other types for a given time period. For 
example, a SAM battery would be weighted high in the first two to three periods and 
low in the remaining. Because SAM’s are a very high threat to aircraft, it makes sense 
to destroy them at the beginning of the war in order to achieve air superiority, hence 
the high initial value. On the other hand, destroying a logistics target at the 
beginning of the war does not make military sense; therefore, they have a low initial 
value and are higher in the last few time periods. 

(5) Coefficient of Confirmability. This number, between zero and one, 
symbolizes the ability of a pilot to assess his battle damage to the target. The 
coefficient mathematically expresses Clausewitz’s "fog of war" principle. If pilots 
cannot confirm kills, or discriminate between live targets and destroyed targets, they 
will waste attacks and munitions on already destroyed targets. The number one 
indicates a pilot is unable to confirm the effect of his first weapons delivery, while zero 
represents perfect target damage assessment [Ref. 7:p. 14]. 

(6) Regeneration Days and Quantity. These specify the average 
number of days required to regenerate a destroyed target type and the number of 
replacements the enemy has. Regeneration is the process by which destroyed targets 
are replaced with available spares (a pontoon bridge for a destroyed spanning bridge) 
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or repaired (filling in the bomb craters on a runway). Additional information may be 
found in [Ref. 8:p. 14]. 



b. Aircraft Data File 



This file (Figure 2-4) contains the aircraft allocated to the theater by 



the Air Force War and Mobilization Plan. There have been as many as 25 different 



types of aircraft. 
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Figure 2-4. Example of an unclassified HEAVY ATTACK Aircraft Data File. 



(1) Selector Number. The SELECTOR file index of this aircraft type. 

(2) Weather Number. The number one indicates the aircraft is all- 
weather radar delivery capable. 

(3) Aircraft Name. Up to four characters, e.g. F15C, A10A. 

(4) Category Label. Each aircraft type is placed into one of nine 
available categories for the output reports. Examples are FALCON and EAGLE. This 
aggregation is used by HEAVY GOAL to identify collateral support requirements. 
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(5) Aircraft Category Number. This identifies the category of each 



aircraft. 



(6) Number of Sorties. Identifies the total number of sorties each 



aircraft type is capable of flying for each time period. 



(7) Attack Mission Percentage. The percentage of sorties devoted 



solely to air-to-ground attack missions for each time period, 
c. Weapon Data File 



All theaters use the same weapon data base; there have been as many 



as 65 weapon types (See Figure 2-5). For planning purposes the allocation of weapons 



among the theaters is decided by the Combat Forces Division. 
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Figure 2-5. Example of an unclassified HEAVY ATTACK Weapon Data File. 



(1) Weapon Number. A numeric index. 

(2) Weapon Name and Description. The name may be up to four 
characters and the description up to 25. An example is LGB — Laser Guided Bomb. 

(3) Play/No Play. This is a binary switch (zero, one). The number one 
removes the weapon from HEAVY ATTACK 

(4) Maximum Expenditure. States the inventory allocated to the 

theater. 
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2. Exclusion Files 



Not every aircraft/target/weapon combination is desirable or feasible in a 
sortie plan. Some convenient method is necessary to exclude particular combinations. 
Of the possible combinations, HEAVY ATTACK accepts inadmissable lists of aircraft 
versus targets, aircraft versus weapons, and aircraft/targets versus weapons. These 
exclusion files are used to preclude combinations which are not militarily feasible. 

The data in each exclusion file is in pairs of rows. For example, in the 
Aircraft versus Target Exclusion File shown in Figure 2-6, the first row in each pair 
is the aircraft type. The following row contains the target types which will never be 
attacked by its paired aircraft type. 



1 

1 2 3 A 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 2A 25 

1 

26 27 28 29 38 AO A1 A5 A7 51 56 57 59 60 61 62 63 6A 66 67 69 71 72 73 7A 
1 

75 76 77 78 79 81 85 

2 

1 2 3 A 5 6 7 8 10 11 12 13 1A 15 16 17 18 19 20 21 22 23 2A 25 26 

• 

• 

13 

1 2 3 A 5 6 7 8 9 10 11 12 13 H 15 16 17 18 19 20 21 22 23 2A 25 

13 

26 27 28 29 38 AO A5 A7 56 59 60 61 62 63 6A 66 67 68 69 70 71 72 73 7A 75 
13 

76 77 78 79 80 81 82 83 8A 85 



Figure 2-6. Example of a HEAVY ATTACK Aircraft versus Target Exclusion 
File. Length for the unclassified data set is 48 lines. 

The Aircraft versus Weapon Exclusion File is similar, except the numbers 
on the following row of each pair are the weapon types which will never be 
expended by its corresponding aircraft type. 
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The remaining exclusion file is the Aircraft/Target versus Weapon 



Exclusion File shown in Figure 2-7. Here, the first row of each pair contains the 
aircraft type followed by the target type. The following row is the weapon types which 

will never be expended by that aircraft against that target. 



1, 5 

2 A 6 8 9 10 11 12 1A 15 17 20 23 2A 25 3A A2 A6 53 55 

1, 5 
57 58 
1,12 

2 A 6 8 9 10 11 12 1A 15 17 20 23 2A 25 26 27 3A 38 A6 

1,12 

A 7 51 53 55 57 58 

• 

• 

99,85 

1 2 3 A 5 6 7 8 9 10 11 12 1A 15 18 19 23 2A A2 A6 

99,85 

A 7 55 57 58 



Figure 2-7. Example of a HEAVY ATTACK Aircraft/Target versus Weapon 
Exclusion File. Length for the unclassified data set is 244 lines. 



D. OUTPUT FROM HEAVY ATTACK 

HEAVY ATTACK produces three reports: short, medium, and long. The Combat 
Forces Division analyst mainly uses the short report (Figure 2-8). It is divided into two 
sections: results for each time period and then cumulative results through all seven 
periods. In the first section, data concerning weapons expended by aircraft categoiy 
and target types destroyed are listed for each time period. The second contains the 
same data as the first, accumulated through the time periods, plus the distribution of 
weapon types to target types. The final data table listed in the report is the 

i 

cumulative and maximum allowable expenditure of weapons used during the "war". 

The medium report is also divided into two sections: results for each time period 
and cumulative results. The first section lists the expected kills of targets by aircraft, 
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Figure 2-8. Example of a HEAVY ATTACK Short Report. Length for the 
unclassified data set is 1500 lines. 



the allocation of aircraft sorties to targets, and the expenditure of weapons by aircraft. 



The second part accumulates the same aircraft allocation and weapons expenditure 



data (see Figure 2-9). 
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Figure 2-9. Example of a HEAVY ATTACK Medium Report. Length for the 
unclassified data set is 2700 lines. 



The long report, typically over 25,000 lines, contains all the information of the 
short and medium reports, plus additional information produced by the optimizer. 
Some additional information is evaluated by the analyst and the theater team, but the 
majority of the information is useful only in diagnosing any data errors identified by 
the optimization. 
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m. HEAVY ATTACK DECISION-MAKING PROCESS 



Analysis of HEAVY ATTACK results is a tedious process. HEAVY ATTACK 
generates a tremendous amount of output. Because there is only one week for the 
planning conference the theater team scarcely has time for a thorough and in-depth 
analysis. In addition, multiple runs of HEAVY ATTACK are required in order to 
generate acceptable results, further exacerbating the problem. 

A. STEPS TOWARD A SOLUTION 

Three steps are employed — analyze the decision-making process, design analysis 
tools, and demonstrate them. The first step examines how the theater teams and 
current Combat F orces Division analyst and operator of HEAVY ATTACK, MAJ Lance 
Buckingham, USAF, make their decisions. Analysis choke points are identified. 
Second, a design is proposed to improve the time and quality of analysis. Finally, this 
design proposal is demonstrated with prototypic computer-based tools. A scenario will 
serve to illustrate the design. 

B. THE DECISION-MAKING PROCESS 

Figure 3-1 shows the HEAVY ATTACK decision-making process. When a theater 
team arrives at the Pentagon for their planning conference, the first step in the 
decision-making process is to create the HEAVY ATTACK input files. These files are 
manually reviewed for correctness. The Best Munitions Table Generator program is 
used to assist in reviewing the exclusion files. Errors in the exclusion files are 
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Figure 3-2. NCAA Decision-Making Process 



corrected, and the Generator program is re-run. This cycle continues until the files 
appear error-free. 

All input files are then passed to HEAVY ATTACK, and the model is run. The 
Combat Forces Division analyst evaluates the output to validate the model. The 
analyst will then present the model’s results to the team, who will decide if any 
adjustments need to be made to the input files. The files are updated and the model 
is re-run. This cycle continues until the team achieves their final results. The team 
may also, during this cycle, send the updated exclusion files back into the Table 
Generator program to re-validate the tables. When the team is satisfied with the 
results from HEAVY ATTACK, their weapon requirements are combined with 
requirements from the other theaters an an input into HEAVY GOAL. 
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Two major analysis problems, or choke points, impede decision-making. First is 



the Table Generator program, which takes significant time to run and to evaluate. 
The second is the actual analysis of HEAVY ATTACK Each of these choke points is 
amenable to automated analysis tools. 
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IV. ANALYSIS TOOLS FOR HEAVY ATTACK 



In order to see how to improve the analysis of HEAVY ATTACK, one must 



understand how the analyst/theater teams presently use the model. Figure 4-1 is a 



current HEAVY ATTACK Aircraft Data File. There is an obvious problem: this file 
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Figure 4-1. HEAVY ATTACK Aircraft Data File. 



contains a lot of numbers with very little associated context. Because HEAVY 
ATTACK is a FORTRAN program, its output may be influenced by a single mis-aligned 
input field. The Target and Weapon Data Files (Figures 2-4 and 2-5) also exhibit this 
weakness. 

The exclusion files also present major problems. Figure 4-2 is a restatement of 
the Air craft /Target versus Weapon Exclusion File. As represented by the figure, these 
files are all numbers with no associated context. For example, the first and second 
lines of Figure 4-2 exclude aircraft type 1 from attacking target type 5 with weapon 
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1, 5 

2 4 6 8 9 10 11 12 14 15 17 20 23 24 25 34 42 46 53 55 

1, 5 
57 58 
1,12 

2 4 6 8 9 10 11 12 14 15 17 20 23 24 25 26 27 34 38 46 

1,12 

47 51 53 55 57 58 

• 

• 

99,85 

1 2 3 4 5 6 7 8 9 10 11 12 14 15 18 19 23 24 42 46 

99,85 

47 55 57 58 



Figure 4-2. Example of an unclassified HEAVY ATTACK Aircraft/Target 
versus Weapon Exclusion File. 

types 2, 4, 6, and so on. The exclusion files are the major source of errors with the 
input. Because of the difficulty in identifying errors in these files, the Table 
Generator program was created. 

After HEAVY ATTACK is run, the output reports are generated. Figure 2-8 is 
just a portion of an unclassified Short Report, which contains information for each of 
the seven time periods plus accumulation across the seven periods. The week-long 
planning conference provides little time to thoroughly analyze the HEAVY ATTACK 
reports. 



A. ANALYSIS IMPROVEMENTS 



The next step was to design a computer system which might provide analysis 
improvements. Several commercially available programs were reviewed and Borland 
International’s Quattro Pro 3.0 spreadsheet program was chosen. Quattro Pro provides 
superior graphic capabilities [Ref. 9,10,11,12] and "prints" to text (ASCII) files. This 
text file capability is extremely important in order to transfer data back and forth 
between desktop computers (Quattro Pro) and mainframes (HEAVY ATTACK). 
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Further, Quattro Pro provides the ability to import a text File and parse the 
information into usable spreadsheet cells. 

The Quattro Pro-based analysis tool has been named QuikDraw. It is divided 
into two sections: Data Entry and Output Listing. 

1. Data Entry 

The Data Entry handles the data inputs for HEAVY ATTACK — 
specifically, the data and exclusion files described in Chapter II. The remaining "other" 
files are updated by the Combat Forces Division analyst using the mainframe 
computer’s text editor. The HEAVY ATTACK Target and Weapon Data Files are 
combined into a single Quattro Pro worksheet. The Aircraft Data File has its own 
Quattro Pro worksheet (Figure 4-3). (The term worksheet is synonymous herein with 
file). 

Currently HEAVY ATTACK lists combinations of aircraft, targets, and 
weapons which are militarily infeasible or invalid. However, the theater teams tend 
to think in terms of valid combinations. Therefore, QuikDraw has, for each HEAVY 
ATTACK exclusion file, a corresponding inclusion worksheet. Now, the users indicate 
with X’s the valid combinations in the worksheets. Figure 4-4 shows the unclassified 
Aircraft versus Target Inclusion Worksheet. 

The Best Munitions Table Generator program, which is the first choke 
point, is replaced by the inclusion worksheets. Hence, there is no need to execute the 
Generator program and analyze the generated tables. 
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1— Indicates all-weather radar delivery capable. 2-Something less 
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Figure 4-3. Example of an unclassified QuikDraw Aircraft Data Worksheet. 
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Figure 4-4. Example of an unclassified QuikDraw Aircraft versus Target 
Worksheet. 
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After a user makes changes to any Data Entry worksheet and is ready to 
re-run HEAVY ATTACK, he executes a Quattro Pro macro program embedded in that 
worksheet. That macro will write the worksheet data to a text file. (For simplicity, 
the macro is executed with the same keystroke in each worksheet). Then, a 
FORTRAN program is executed to convert the text files into the format required for 
HEAVY ATTACK This is necessary because HEAVY ATTACK as a FORTRAN 
program, requires its input data to be in specific columns. 

2. Output Listing 

The Output Listing section has four worksheets; they are listed in Table 
I. Within each worksheet are three graph types: single run, multiple nan, and 
cumulative. The single run graphs display information for the current run; multiple 
graphs present information from two successive nans. The cumulative graphs display 
"course of the war" information. All graphs display percentages, with raw numbers 
available below the graphs. 

Table I. QUIKDRAW OUTPUT WORKSHEETS. THESE WORKSHEETS ARE 
CAPABLE OF GENERATING THREE TYPES OF GRAPHS. EXAMPLES OF THESE 
GRAPH TYPES ARE SHOWN IN FIGURES 4-5 THROUGH 4-8. 

Aircraft Attack Missions Flown by 'rime Period 
Weapon Types Expended by Time Period 
Weapon Types Expended across Time Periods 
Target Types Destroyed across Time Periods 
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There is a fundamental difference between the "by time period" and the 
"across time period" worksheets. For the former the graphed values are the 
percentages for each time period. Thus, the graphed values sum to 100 percent. In 
the latter case, the graphed values are the percentages over all time periods, and may 
exceed 100 percent due to target regeneration. 

A special HEAVY ATTACK report (DWUNCL) was created to export data 
to QuikDraw. This report contains the data necessary to generate the graphs, plus 
some specific syntactic semaphores for the Quattro Pro parsing function. The parsing 
semaphores allow Quattro Pro to identify only the necessary columns from a report 
table, significantly reducing the amount of data kept in the worksheets. 

Figures 4-5 and 4-6, Attack Missions Flown and Weapons Expended, are 
examples of "by time period" graphs. With these graphs the analyst and the theater 
team can instantly see how the aircraft were allocated, or weapons expended in time 
period one. This analysis takes minutes rather than the tens of minutes or hours 
required to analyze the current reports. In addition this analysis is more thorough. 
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Period 1 
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Figure 4-5. QuikDraw current run graph of Aircraft Attack Missions Flown by 
Time Period for time period 1 using the unclassified data set. Percentages sum 
to 100. 



Period 1 
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Figure 4-6. QuikDraw current run graph of Weapons Expended By Time 
Period for time period 1 using the unclassified data set. Percentages sum to 
100 . 
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Another feature of QuikDraw is the ability to view multiple runs. Figure 
4-7 is an example of a multiple run graph, showing how the expenditure of weapons 
is affected by removing an aircraft from the data set. To produce this graph, HEAVY 



Period 1 




ESS last CURRENT 



Figure 4-7. QuikDraw multiple run graph of Weapons Expended Across Time 
Periods for time period 1 using the unclassified data set. 



ATTACK fust executed with thirteen aircraft types in the data set. The weapons 
expended by those aircraft are represented by light-shaded, double cross-hatch bars 
labeled Last (run). Aircraft AC6, having flown about thirteen percent of the missions 
in the fust rim, was removed from the data set and the model re-run. The weapon 
types expended by the remaining twelve aircraft are represented by the heavy-hashed 
bars labeled Current (run). The analyst/theater teams are then able to analyze how 
removing an aircraft from HEAVY ATTACK affect the expenditure of weapons. With 
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QuikDraw the analysis takes minutes, rather than the hours analyzing the HEAVY 
ATTACK reports by hand. 

Finally, QuikDraw allows the analyst/ theater team to view cumulative 
results over "the course of the war" (all seven time periods). An example is 
Figure 4-8 showing how the target data base was destroyed during the "war". Here five 
of 38 targets had higher than 100 percent destuction because of HEAVY ATTACK’S 
target regeneration feature. 

B. TESTING AND IMPLEMENTATION 

QuikDraw was tested using three different subsets from an unclassified data set 
provided by the United States Air Force Combat Forces Division. The tests generated 
high confidence that Data Entry worksheets provide HEAVY ATTACK with relatively 
quick-entry and error-free input. The tests also generated all three graph types for 
the four Output Listing worksheets. 

QuikDraw consists of 12 files, 24 graphs, nearly 178,000 spreadsheet cells, and 
1,100 lines of Quattro Pro macro coding using nearly 1,900 macro instructions. The 
FORTRAN code to convert the QuikDraw Data Entry worksheets into the HEAVY 
ATTACK input files consists of 581 lines. QuikDraw is available from the author or 
Professor Bradley. 
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Figure 4-8. QuikDraw cumulative graph of Targets Destroyed Across Time 
Periods for time periods 1—7 using the unclassified data set. Total percentages 
may exceed 100% of available target types due to regeneration. 
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V. CONCLUSION 



HEAVY ATTACK is an important operations research model. It has been used 
for over 20 years and may continue to be used in the foreseeable future. The 
improvements in analyzing this model offer significant savings in man-hours and 
money. 

HEAVY ATTACK is just one example of using a spreadsheet as a tool to analyze 
output. The approach developed here to (1) analyze a decision-making process, (2) 
design analysis tools, and (3) demonstrate them, can be applied to other large-scale 
optimization models. 

This thesis designed and implemented a state-of-the-art output analysis system 
that demonstrates analysis choke points can be reduced or eliminated. The current 
cumbersome, time-consuming data input and quality assurance process is replaced by 
QuikDraw which does the same work in significantly less time, reduces errors in the 
input files, and minimizes the time spent analyzing hundreds of sheets of output 
through the use of graphics. In total this improves the quality of analysis. 

A preliminary version of QuikDraw was demonstrated to the United States Air 
Force Combat Forces Division in Washington D.C. in December 1991. As a result of 
the demonstration, they intend to incorporate QuikDraw into the next generation 
NCAA process, which will be based on a 486 personal computer. 
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